catch common resource schema issues in cfn validate #785

PatMyron · 2021-06-22T03:34:11Z

how to run new validations on all existing resource provider schemas

encouraging explicit boolean values instead of defaults:

Explicitly specify value for taggable: 314 violations out of 328 public AWS resource types

Explicitly specify value for insertionOrder for array: 724 violations out of 897 arrays:

   47 AWS::Kendra::DataSource
   32 AWS::ECS::TaskDefinition
   28 AWS::CloudFront::Distribution
   17 AWS::QuickSight::DataSet
   17 AWS::EC2::NetworkInsightsAnalysis
   16 AWS::WAFv2::WebACL
   16 AWS::WAFv2::RuleGroup
   15 AWS::QuickSight::Dashboard
   15 AWS::QuickSight::Analysis
   15 AWS::ApplicationInsights::Application
   13 AWS::EC2::SpotFleet
   10 AWS::QuickSight::Template
   10 AWS::ElasticLoadBalancingV2::ListenerRule
    8 AWS::ResourceGroups::Group
    8 AWS::MediaPackage::OriginEndpoint
    8 AWS::KinesisFirehose::DeliveryStream
    8 AWS::IoTEvents::DetectorModel
    8 AWS::ECS::Service
    7 AWS::SageMaker::MonitoringSchedule
    7 AWS::QuickSight::Theme
    7 AWS::MediaPackage::PackagingConfiguration
    7 AWS::ImageBuilder::DistributionConfiguration
    7 AWS::AppFlow::Flow
    6 AWS::SageMaker::ModelQualityJobDefinition
    6 AWS::SageMaker::DataQualityJobDefinition
    6 AWS::LookoutMetrics::AnomalyDetector
    6 AWS::Lambda::Function
    6 AWS::IoT::TopicRule
    6 AWS::DataBrew::Recipe
    6 AWS::Budgets::BudgetsAction
    5 AWS::ServiceCatalog::CloudFormationProvisionedProduct
    5 AWS::QuickSight::DataSource
    5 AWS::NimbleStudio::LaunchProfile
    5 AWS::Lambda::EventSourceMapping
    5 AWS::GreengrassV2::ComponentVersion
    5 AWS::GameLift::Fleet
    5 AWS::FMS::Policy
    5 AWS::FIS::ExperimentTemplate
    5 AWS::DataBrew::Dataset
    5 AWS::Config::ConfigurationAggregator
    5 AWS::AuditManager::Assessment
    4 AWS::SageMaker::ModelExplainabilityJobDefinition
    4 AWS::SageMaker::ModelBiasJobDefinition
    4 AWS::SageMaker::Domain
    4 AWS::SSM::Document
    4 AWS::Redshift::Cluster
    4 AWS::OpsWorksCM::Server
    4 AWS::NimbleStudio::StudioComponent
    4 AWS::Kendra::Index
    4 AWS::GroundStation::DataflowEndpointGroup
    4 AWS::ElasticLoadBalancingV2::Listener
    4 AWS::ElastiCache::GlobalReplicationGroup
    4 AWS::EKS::FargateProfile
    4 AWS::ECS::TaskSet
    4 AWS::ECS::Cluster
    4 AWS::EC2::EC2Fleet
    4 AWS::CustomerProfiles::Integration
    4 AWS::ACMPCA::Certificate
    3 AWS::Synthetics::Canary
    3 AWS::SageMaker::UserProfile
    3 AWS::SSO::InstanceAccessControlAttributeConfiguration
    3 AWS::SSM::Association
    3 AWS::S3Outposts::Bucket
    3 AWS::Macie::FindingsFilter
    3 AWS::IoT::DomainConfiguration
    3 AWS::IAM::OIDCProvider
    3 AWS::GameLift::GameServerGroup
    3 AWS::Events::Connection
    3 AWS::DynamoDB::GlobalTable
    3 AWS::DataSync::Task
    3 AWS::DataBrew::Job
    3 AWS::CustomerProfiles::ObjectType
    3 AWS::CodeGuruProfiler::ProfilingGroup
    3 AWS::CodeArtifact::Repository
    3 AWS::CloudWatch::MetricStream
    3 AWS::CloudWatch::CompositeAlarm
    3 AWS::CloudFront::OriginRequestPolicy
    3 AWS::CloudFront::CachePolicy
    3 AWS::Backup::BackupPlan
    3 AWS::AppRunner::Service
    3 AWS::AppIntegrations::EventIntegration
    2 AWS::WorkSpaces::ConnectionAlias
    2 AWS::WAFv2::RegexPatternSet
    2 AWS::WAFv2::IPSet
    2 AWS::StepFunctions::StateMachine
    2 AWS::SageMaker::Project
    2 AWS::SageMaker::FeatureGroup
    2 AWS::SageMaker::AppImageConfig
    2 AWS::SSMContacts::Contact
    2 AWS::SSM::ResourceDataSync
    2 AWS::SES::ContactList
    2 AWS::Route53::HealthCheck
    2 AWS::MediaPackage::Channel
    2 AWS::MediaPackage::Asset
    2 AWS::MediaConnect::FlowVpcInterface
    2 AWS::Macie::CustomDataIdentifier
    2 AWS::MWAA::Environment
    2 AWS::LicenseManager::License
    2 AWS::LicenseManager::Grant
    2 AWS::IoTSiteWise::Gateway
    2 AWS::IoTEvents::Input
    2 AWS::IoT::TopicRuleDestination
    2 AWS::ImageBuilder::InfrastructureConfiguration
    2 AWS::ImageBuilder::ImageRecipe
    2 AWS::ImageBuilder::ContainerRecipe
    2 AWS::IAM::VirtualMFADevice
    2 AWS::GroundStation::MissionProfile
    2 AWS::GlobalAccelerator::EndpointGroup
    2 AWS::GlobalAccelerator::Accelerator
    2 AWS::EFS::FileSystem
    2 AWS::ECS::ClusterCapacityProviderAssociations
    2 AWS::ECR::ReplicationConfiguration
    2 AWS::EC2::PrefixList
    2 AWS::DataSync::Agent
    2 AWS::DataBrew::Schedule
    2 AWS::Config::OrganizationConformancePack
    2 AWS::CloudFront::RealtimeLogConfig
    2 AWS::CUR::ReportDefinition
    2 AWS::Backup::BackupSelection
    2 AWS::ApiGateway::DomainName
    2 AWS::ApiGateway::ApiKey
    2 AWS::Amplify::Branch
    2 AWS::ACMPCA::CertificateAuthority
    1 AWS::XRay::SamplingRule
    1 AWS::XRay::Group
    1 AWS::Timestream::Table
    1 AWS::Timestream::Database
    1 AWS::Signer::SigningProfile
    1 AWS::ServiceCatalog::ServiceAction
    1 AWS::SageMaker::Pipeline
    1 AWS::SageMaker::ModelPackageGroup
    1 AWS::SageMaker::Image
    1 AWS::SageMaker::DeviceFleet
    1 AWS::SageMaker::Device
    1 AWS::SageMaker::App
    1 AWS::SSMIncidents::ResponsePlan
    1 AWS::SSMIncidents::ReplicationSet
    1 AWS::S3Outposts::Endpoint
    1 AWS::Route53Resolver::FirewallDomainList
    1 AWS::Route53::HostedZone
    1 AWS::NimbleStudio::StreamingImage
    1 AWS::NetworkManager::Site
    1 AWS::NetworkManager::Link
    1 AWS::NetworkManager::GlobalNetwork
    1 AWS::NetworkManager::Device
    1 AWS::NetworkFirewall::Firewall
    1 AWS::MediaPackage::PackagingGroup
    1 AWS::MediaConnect::FlowOutput
    1 AWS::MediaConnect::FlowEntitlement
    1 AWS::Logs::QueryDefinition
    1 AWS::Lambda::CodeSigningConfig
    1 AWS::Kendra::Faq
    1 AWS::IoTWireless::WirelessGateway
    1 AWS::IoTWireless::WirelessDevice
    1 AWS::IoTWireless::TaskDefinition
    1 AWS::IoTWireless::ServiceProfile
    1 AWS::IoTWireless::PartnerAccount
    1 AWS::IoTWireless::DeviceProfile
    1 AWS::IoTWireless::Destination
    1 AWS::IoTSiteWise::Project
    1 AWS::IoTCoreDeviceAdvisor::SuiteDefinition
    1 AWS::IoT::ProvisioningTemplate
    1 AWS::IoT::Authorizer
    1 AWS::ImageBuilder::Component
    1 AWS::IAM::ServerCertificate
    1 AWS::IAM::SAMLProvider
    1 AWS::GroundStation::Config
    1 AWS::Glue::Schema
    1 AWS::Glue::Registry
    1 AWS::GlobalAccelerator::Listener
    1 AWS::ElastiCache::UserGroup
    1 AWS::ElastiCache::User
    1 AWS::EMR::Studio
    1 AWS::EFS::AccessPoint
    1 AWS::ECS::CapacityProvider
    1 AWS::EC2::TransitGatewayPeeringAttachment
    1 AWS::EC2::TransitGatewayMulticastDomain
    1 AWS::EC2::TransitGatewayConnect
    1 AWS::EC2::TransitGateway
    1 AWS::EC2::FlowLog
    1 AWS::DevOpsGuru::ResourceCollection
    1 AWS::Detective::Graph
    1 AWS::DataSync::LocationSMB
    1 AWS::DataSync::LocationObjectStorage
    1 AWS::DataSync::LocationNFS
    1 AWS::DataSync::LocationFSxWindows
    1 AWS::DataSync::LocationEFS
    1 AWS::DataBrew::Project
    1 AWS::CustomerProfiles::Domain
    1 AWS::Config::StoredQuery
    1 AWS::Config::ConformancePack
    1 AWS::CodeStarConnections::Connection
    1 AWS::CodeGuruReviewer::RepositoryAssociation
    1 AWS::CodeArtifact::Domain
    1 AWS::CloudFront::KeyGroup
    1 AWS::CloudFormation::StackSet
    1 AWS::Chatbot::SlackChannelConfiguration
    1 AWS::Cassandra::Table
    1 AWS::Cassandra::Keyspace
    1 AWS::Backup::BackupVault
    1 AWS::Athena::WorkGroup
    1 AWS::Athena::DataCatalog
    1 AWS::ApiGateway::ClientCertificate

This would introduce significantly more cfn validate violations than the rest of the validations combined

Yunhao-Jiao · 2021-06-22T17:40:57Z

btw looks like one build failed

encouraging explicit boolean values instead of defaults

PatMyron · 2021-06-25T02:58:19Z

Current running totals for THIRD_PARTY resource provider schemas:

 136 CloudFormation properties don't usually start with lowercase letters (mostly Spot::Elastigroup::Group)
  56 Explicitly specify value for insertionOrder for array
  42 Explicitly specify value for taggable
   2 [Warning] Resource spec validation would fail from next major version. Provider should mark additionalProperties as false if the property is of object type and has properties or patternProperties defined in it. Please fix the warnings: 'additionalProperties' is a required property
   2 non-ASCII characters found in resource schema

Script to download THIRD_PARTY resource provider schemas to current directory (modified from aws-cloudformation/cfn-lint#1732)

import boto3
for page in boto3.client('cloudformation').get_paginator('list_types').paginate(
    Visibility='PUBLIC',
    Type='RESOURCE',
    Filters={
        'Category': 'THIRD_PARTY',
    },
):
    for resource_type in page['TypeSummaries']:
        with open(resource_type['TypeName'].replace("::", "-").lower() + '.json', 'w') as f:
            print(boto3.client('cloudformation').describe_type(Arn=resource_type['TypeArn'])['Schema'], file=f)

…on#785) encouraging explicit boolean values instead of defaults

PatMyron requested review from Yunhao-Jiao, omkhegde, ammokhov and aygold92 June 22, 2021 03:35

sungkkim approved these changes Jun 22, 2021

View reviewed changes

Yunhao-Jiao approved these changes Jun 22, 2021

View reviewed changes

PatMyron force-pushed the validate branch from d4ee256 to 54d9350 Compare June 22, 2021 18:57

PatMyron added 2 commits June 22, 2021 14:29

catch common resource schema issues in cfn validate

5c4830d

encouraging explicit boolean values instead of defaults

adding explicit taggable value to test resource schema

9dad5d5

PatMyron force-pushed the validate branch from 54d9350 to 9dad5d5 Compare June 22, 2021 21:30

PatMyron merged commit b649f00 into master Jun 22, 2021

PatMyron deleted the validate branch June 22, 2021 21:37

PatMyron added schema Related to the provider meta schema schema processing labels Jun 25, 2021

PatMyron mentioned this pull request Aug 2, 2021

cfn validate iann0036/cfn-rps-lint#9

Open

kddejong pushed a commit to kddejong/cloudformation-cli that referenced this pull request Oct 24, 2022

catch common resource schema issues in cfn validate (aws-cloudformati…

c24ace7

…on#785) encouraging explicit boolean values instead of defaults

kddejong pushed a commit to kddejong/cloudformation-cli that referenced this pull request Oct 24, 2022

catch common resource schema issues in cfn validate (aws-cloudformati…

b885a18

…on#785) encouraging explicit boolean values instead of defaults

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

catch common resource schema issues in cfn validate #785

catch common resource schema issues in cfn validate #785

PatMyron commented Jun 22, 2021 •

edited

Loading

Yunhao-Jiao commented Jun 22, 2021

PatMyron commented Jun 25, 2021 •

edited

Loading

catch common resource schema issues in cfn validate #785

catch common resource schema issues in cfn validate #785

Conversation

PatMyron commented Jun 22, 2021 • edited Loading

Yunhao-Jiao commented Jun 22, 2021

PatMyron commented Jun 25, 2021 • edited Loading

PatMyron commented Jun 22, 2021 •

edited

Loading

PatMyron commented Jun 25, 2021 •

edited

Loading